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In this study we explored the effect of notations that undergraduates produce on their 
reasoning abilities. Participants solved four analytical reasoning problems, making 
notes to solve two problems and solving two without notations. Most participants 
produced notes when given the opportunity. The production of higher quality notes 
did not aid performance when we compared the write and no write conditions and 
those producing poorer quality notes performed even worse in the write condition. 
The results suggest that students do not necessarily have adequate note-taking skills 
and that they may need instruction in the production and use of external 
representations. 
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Dans cette etude, les auteures ont explore Teffet des notes prises par des etudiants de 
premier cycle sur leur capacite de raisonnement. Les participants ont resolu quatre 
problemes de raisonnement analytique, prenant des notes en vue de resoudre deux 
problemes sur quatre. La plupart des participants ont pris lorsqu'ils en avaient 
Toccasion. La prise de meilleures notes n'a pas contribue a Tobtention de meilleurs 
resultats si Ton compare les contextes avec ou sans notes ; les personnes qui ont pris 
de moins bonnes notes ont obtenu un rendement encore plus mauvais dans le 
contexte avec notes. Les resultats semblent indiquer que les etudiants ne savent pas 
necessairement comment prendre de bonnes notes et qu'il faudrait peut-etre leur 
montrer comment prendre des notes et s'en servir. 

Mots cles : resolution de problemes, prise de notes, qualite des notes 
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As writing systems developed in Western culture, they were 
accompanied by large social changes including more complex 
agricultural and trade systems, political and economic advances, and 
mass education and production (Scribner & Cole, 1981). Although such 
changes have played an important role in modern society, scholars have 
long debated the impact of literacy on both the individual and society 
and its relationship with cognitive processes. For example, Aristotle felt 
that external representation of informafion would nof affecf fhoughf 
because if was only a franscripfion of speech (Olson, 1994), a view sfill 
held by some fheorisfs foday (e.g., Carrufhers, 1990). This perspecfive 
assumes fhaf liferacy does nof impacf individual cognifion, alfhough if 
can have an influence af fhe sociefal level. In confrasf, ofher fheorisfs 
have suggesfed fhaf fhe acf of wrifing may alfer fhoughf processes 
(Harris, 1989). In fhis view, liferacy paved fhe way for fhe developmenf 
of logical, analyfical, and scienfific fhinking (Donald, 1991; Goody & 
Waft, 1968; Havelock, 1963; McLuhan, 1962; Ong, 1982). It can have an 
impact at the individual level. 

The effect of prinf exposure on cognifive processes is one area fhaf 
examines fhe impacf of liferacy on fhe individual. In fhe field of fhe 
cognifive psychology of reading, Sfanovich and Cunningham (1992) 
have affempfed fo pinpoinf fhe cognifive processes fhaf supporf effecfive 
reading performance. In confrasf, fhere has been liffle focus on fhe 
influence of exposure fo prinf on cognifive processes. Avid readers fend 
fo differ from nonreaders in a variety of ways, including fheir cognifive 
skills, behavioural habifs, and background variables (Stanovich, 1993). 
Sfanovich, Wesf, and Harrison (1995) also found fhaf prinf exposure was 
direcfly relafed fo measures of crysfallized intelligence (i.e., vocabulary, 
general knowledge), but they did not find a direcf influence of prinf 
exposure on fluid abilifies (i.e., reasoning, working memory). 

The debafe abouf fhe impacf of liferacy remains, parfly because of 
fhe way if has been defined. Mosf offen liferacy is fhoughf of as fhe 
abilify fo read and wrife. However, fhis definifion limifs fhe 
inferprefafion of liferacy and excludes ofher imporfanf funcfions. A 
more comprehensive definifion of liferacy includes fhe assumpfions, 
expecfafions, and affifudes fhaf accompany fhe basic skills of reading 
and wrifing (Illich, 1991; McLane & McNamee, 1990; Olson, 1994). 
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Limiting the definition of liferacy fo fhe skills of reading and wrifing 
only confirms fhe implicafions of liferacy (Olson, 1994) and fhe research 
fhaf is conducfed. Alfhough liferacy is offen viewed as compefence in 
alphabefic reading (Donald, 1991), a number of ofher symbol sysfems 
exisf. A more comprehensive definifion of liferacy is needed which 
should include fhe abilify fo use exfernal represenfafions in general and 
nof fo be resfricfed fo alphanumeric symbols. Exfernal represenfafions, 
such as diagrams, graphs, or picfures are commonly used in problem 
solving, decision making, or reasoning fasks. If is fheorized fhaf exfernal 
represenfafions are an infegral parf of fhese fasks (Zhang, 1997). 
However, until recenfly no distinction has been made between internal 
and external representations, which work together in many cognitive 
tasks (Zhang, 1997). This distinction between internal and external 
representations allows for fhe sfudy of exfernal represenfafions as a field 
on fheir own. The presenf sfudy examines how undergraduafes use 
exfernal represenfafions fo aid cognitive processing, specifically 
performance on a reasoning fask. 

Theorisfs from a number of viewpoints, including historical, 
evolutionary, and cross-cultural perspectives, have noted that a main 
property of exfernal represenfafions is to serve as a memory aid. Further 
research has indicated that external memory aids are used more often 
and they are perceived to be more reliable than internal memory aids 
(Harris, 1982; Intons-Peterson & Fournier, 1986). Given this property of 
nofafions, several sfudies have examined fhe impacf of exfernal symbols 
on memory (for review see Kiewra, 1985, 1989). 

Eskriff, Lee, and Donald (2001) conducfed a series of experimenfs 
examining fhe impacf of producing nofafions on fhe memory 
performance. Parficipanfs who played a memory card game were 
allowed fo make notes to help in some conditions but not in others. 
When given the opportunity to make notes, over three-quarters of fhe 
parficipanfs chose fo do so, and fhis nofe producfion improved memory 
performance. Furfhermore, fhey found fhaf fhe unexpecfed removal of 
nofafions led fo a decrease in performance, suggesfing fhaf fhe 
sponfaneous producfion of nofes served as exfernal memory sforage. 
However, furfher experimenfafion found fhaf fhe idenfify of cards was 
still refained in memory, fhough nofafions were used to store location 
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information for cards (Eskritt et al., 2001). These findings suggest that 
the relationship between memory and literacy is more complex than 
typically thought. These findings warrant further study. 

Theorists also believe that notations may influence the cognitive 
process of reasoning (Goody & Watt, 1968; Ong, 1982); however, there 
has been little empirical research examining the effects of notations on 
problem solving and reasoning. Goody and Watt (1968) have argued 
that formal logic and reasoning began in Ancient Greece as a result of the 
development of a literate society. In fact, Levy-Bruhl (1923) made the 
assumption that oral societies were "prelogical" in their thinking. A 
more modern standpoint is that oral cultures use similar logic principles 
as literate cultures, with the difference being how societies generalize 
information (Luria, 1976). For example, in his research with the illiterate 
adult residents of an isolated area of Russia, Luria (1976) found that 
schooling could have a dramatic effect on the responses given to 
syllogisms. Luria argued that the introduction of literacy led to an 
increase in abstract, theoretical, and logical thinking compared to 
residents who had not received schooling. 

Zhang has examined how different types of external representations 
can affect a literate individual's problem solving, using variations of two 
games, Tic-Tac-Toe (Zhang, 1997) and the Tower of Hanoi (Zhang & 
Norman, 1994). He found that the type of external representation used 
to characterize a problem could influence participants' perceived 
difficulty of the problem as well as their performance on the problem. 
The form of the external representation can influence what information 
participants perceive more readily, and what processes they use in 
solving problems (Zhang, 1997). Furthermore, from the problem solver's 
point of view, tasks are completely different with and without external 
representations (Zhang & Norman, 1994). People often use external 
representations as support when solving problems (Gary & Garlson, 
2001). External representations can be used to maintain and manage 
information necessary in solving a problem and in clarifying the steps 
needed in solving the problem (Gary & Garlson, 1999; Gox, 1999). 
However, Gary and Garlson (2001) have suggested that external 
representations are helpful only if the benefits of using them outweigh 
the costs. For example, when using external representations, participants 
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need to attend not only to the task at hand but also to the external 
representations that they employed as an aid (Carlson, 1997; Miyake & 
Shah, 1999). 

Another issue is participants' production of their own notations for 
reasoning fasks. Cox (1999) found fhaf self-consfrucfed nofafions can be 
eifher graphical or linguisfic and fhaf differenf individuals consfrucf a 
wide range of represenfafions. Furfhermore, Cox and Bma (1995) found 
fhaf fhe majority of subjecfs (> 80%) use some form of nofafions to aid in 
solving analytical reasoning tasks. However, when studies have 
examined how participants use self-generated or presented 
representations to solve problems, they did not compare participants' 
performance on problems solved wifh and wifhouf fhe use of nofafions 
(Cox & Brna, 1995; Zhang, 1997). Conditions where parficipanfs do nof 
make nofes can serve as a baseline for individual performance. 
Providing parficipanfs wifh fhe opporfunify fo use nofes and nof use 
nofes allows researchers fo make comparisons befween fhe fwo 
condifions fo see if fhe use of nofafions acfually aids performance. 

THE STUDY 

Previous research has shown fhaf schooling may have an effecf on 
reasoning skills (Luria, 1976) and fhe types of exfemal represenfafions 
provided by an experimenfer may affecf problem solving (Zhang, 1997; 
Zhang & Norman, 1994). The presenf sfudy explored fhe effecf of 
nofafions fhaf undergraduafe sfudenfs produce have on fheir analytical 
reasoning abilifies. We gave parficipanfs four analyfical reasoning fasks 
fo complefe, fwo of which fhey had fhe option fo make nofes fo solve, 
and fhe ofher fwo where fhey did nof have fhe opporfunify to produce 
notations. We then divided participants into groups based on the quality 
of fhe nofafions produced and we compared fheir performance on 
analyfic questions when fhey had fhe opporfunify fo make nofes and 
when fhey did nof. 

Specifically, fhe sfudy examined fhree questions. We invesfigafed 
whefher undergraduafes produced nofes fo aid in a reasoning fask when 
given fhe opporfunify, and whaf fypes of nofafions fhey chose fo 
produce. Based on previous research, we expecfed fhaf fhe majorify of 
parficipanfs would choose fo make nofafions when given fhe 
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opportunity (e.g., Cox & Brna, 1995; Eskritt et al, 2001). Given the results 
of research on the effects of differenf fypes of exfemal represenfafions on 
problem solving (Zhang, 1997; Zhang & Norman, 1994), we knew fhaf 
parficipanfs mighf nof produce nofes of similar qualify and fhaf fhis 
difference in qualify mighf affecf fhe usefulness of fhe nofafions. We also 
examined whefher fhe use of nofafions improved fhe performance of 
parficipanfs as measured by fhe number of correcf responses when fhey 
make nofes compared to when they do not. We predicted that 
performance would improve when parficipanfs make nofafions fo aid in 
solving fhe reasoning problems. Finally, we examined parficipanfs' 
exposure to print. Research has found fhaf familiarity of represenfafions 
is relafed fo reasoning wifh nofes (Cox 1999) and fhe type of 
represenfafion used affecfs fhe level of difficulfy of a problem (Zhang & 
Norman, 1994). Thus, we predicfed fhaf prinf exposure would be 
posifively correlafed wifh performance on fhe reasoning problems 
because a greafer familiarify wifh reading could lead fo a greafer 
familiarify wifh fhe possible nofafions fhaf parficipanfs could use fo 
solve problems. 

METHOD 

Participants 

Forfy-six undergraduafe sfudenfs enrolled in infroducfory psychology 
parficipafed. Forfy participants were female and six were male. The 
mean age of parficipanfs was 22.4 (SD = 6.7). They received a bonus 
mark in fhe course in refurn for fheir parficipafion. Parficipanfs were 
freafed efhically according fo Tri-Council guidelines (Canadian Public 
Works and Govemmenf Services, 1998). 

Materials and Procedure 

Each parficipanf was fesfed individually. We presenfed parficipanfs 
wifh four analyfical reasoning problems in paper formal. Problems 
were adapfed from sample Graduafe Record Examination (GRE) 
quesfions (Brownsfein, Weiner, Green & Hilberf, 1990; GRE practice fesf, 
1996). Each problem consisfed of a passage involving six ifems followed 
by fhree resfricfions. Parficipanfs were required fo answer fwo multiple- 
choice quesfions based on fhe passage and fhe resfricfions. (See 
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Appendix for a sample problem.) We conducted pilot testing to ensure 
that responders could complete each question without the use of writing. 
In the pilot study, seven undergraduate students attempted to solve ten 
different questions. We chose four questions because pilot test students 
demonstrated that they could complete them without making notes, 
although the questions were difficult enough to warrant making notes. 

There were two conditions under which participants solved 
problems: the write condition and the no write condition. This factor was 
within-subject as participants completed both conditions. In the write 
condition, we allowed participants to make notes. They were told, "You 
can use the pencil and paper provided to help you complete the 
problems." In the no write condition, we required participants to orally 
answer the questions and to solve the question without the aid of pencil 
and paper. The order of questions and writing conditions was 
counterbalanced. 

After the completion of the four questions, participants were given 
the Author Recognition Test (ART) as a measure of their exposure to 
print. Stanovich (1993) found that the ART was a valid measure of print 
exposure because scores on the ART correlated positively with other 
measures of exposure to print such as interview techniques and activity 
diary methods. The ART is a simple checklist including the names of 43 
authors and 43 foils (Stanovich, 1993). We instructed participants to 
place a mark next to the names on the list they were sure were authors. 
We calculated participants' scores by subtracting the number the foils 
incorrectly chosen from the number of authors correctly identified. 

Scoring 

We evaluated the quality of notations produced by the participants by 
examining the amount and type of information contained within the 
notes. Notes were scored on a scale from 0 to 4. AO indicated that only 
an answer was given, whereas 1 indicated that very little was written 
and it was not obvious how the information in the notes could help solve 
the problem. For example, some participants simply rewrote the items 
contained in the question. A score of 2 was awarded when the 
restrictions were rewritten in a new format or incorporated into a 
diagram, but no new information was included. A 3 indicated that the 
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restrictions were rewritten in a new format or incorporated in a diagram 
so that some new information could be derived. A 4 indicated that not 
only was new information provided, but also the question was almost or 
totally solved within the notations. Accuracy of the notations was also 
examined but none of fhe parficipanfs made any errors in fheir nofafions. 
Two rafers rafed all fhe nofafions and resolved any differences in rafings. 
Infer-rafer reliability was 89.7 per cent. 

Figure 1 illustrates the frequency of scores parficipanfs received for 
fheir nofafions. Parficipanfs were divided info fwo groups, fhose who 
made poor qualify nofafions and fhose who made good qualify 
nofafions, based on fhe mean qualify of fhe nofafions scores for bofh 
quesfions in fhe wrife condifion. Those parficipanfs whose mean score 
was less fhan 1.5 were placed in fhe poor nofafional qualify group {n = 
17, M score = .93, SD = .6) while fhose parficipanfs whose mean score was 
more fhan 1.5 were labeled fhe good nofafional qualify group {n = 21, M 
score = 2.2, SD = .6). The poor nofafional qualify group confained fhose 
parficipanfs whose nofes were minimal (i.e., resfafed fhe ifems in fhe 
quesfion) and fhe information confained wifhin fhe nofes was of 
quesfionable aid for solving fhe problem. The good nofafional qualify 
group confained fhose parficipanfs who produced more subsfanfial 
nofafions fo aid in solving fhe problems. See Figure 2 for examples of 
nofafions for each group. 

RESULTS 

Five parficipanfs chose nof fo make nofes when given fhe opporfunify 
(10.9%) and fhree parficipanfs (6.5%) chose fo make nofes only for one of 
fhe fwo quesfions in fhe wrife condifion. We excluded fhese parficipanfs 
from furfher analysis, leaving us wifh 38 parficipanfs from whom we 
used dafa in fhe analyses. 

Performance Measures 

We conducfed fwo ANOVAs fo examine performance on fhe analytical 
reasoning quesfions. Preliminary analyses found no difference in fhe 
order of fhe wrife and no wrife condifions (F(l,36) = 1.27, n.s, for number 
correcf and f(l,36) = 0.15, n.s, for time); fherefore we collapsed fhe dafa 
across fhis variable. In fhe firsf analysis, we examined fhe number of 




Analytical Reasoning Skills 


863 



Notational Score 

Figure 1 Frequency of notation scores for participants who wrote 


A. 


B. 
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Figure 1 Example of: a) a poor quality notation and b) a good quality notation. 
Poor notations contained information that was of questionable use for solving 
the problem, while good notations contained more substantial information that 
could aid in problem solving 
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correct responses participants made. The good notational quality group 
had a mean of 3.2 (SD = .9) questions correct in the write condition and 
3.0 (SD = 1.0) questions correct in the no write condition. For the poor 
notational quality group, the mean was 2.4 (SD = 1.4) questions correct in 
the write condition and 2.9 (SD = 0.8) in the no write condition. We 
performed a 2 (condifion) X 2 (nofafional qualify group) ANOVA fo 
examine whefher fhe number of correcf responses in fhe wrife condifion 
differed significanfly from fhe no wrife condifion depending on 
nofafional qualify. Resulfs indicafed a significanf inferacfion befween 
fhe good and poor nofafional qualify groups and condifion on fhe 
number of correcf responses, f(l, 36) = 5.03, p < .05, rf = .12. An analysis 
of simple effecfs revealed a sfafisfically significanf difference in fhe 
number of correcf responses in fhe wrife condifion befween fhe fwo 
groups (p < .05). The good nofafional qualify group (M = 3.1, SD = 1.0) 
did significanfly better fhan fhe poor nofafional qualify group (M = 2.7, 
SD = 1.2). Alfhough fhose who made poor qualify nofafions fended fo 
have fewer questions correcf when given fhe opporfunify fo write, this 
difference was nof significanf, p = .07. However, nofafional production 
did nof appear fo aid fhe good nofafional qualify group because fhe 
number of correcf responses for fhese parficipanfs did nof differ befween 
condifions (wrife condifion: M = 3.2, SD = 0.9; no wrife condifion: M = 
3.0, SD = 1.0). There was also no sfafisfically significanf difference found 
befween groups for fhe no wrife condifion. 

The second analysis was a 2 (wrife versus no wrife condifion) X 2 
(poor versus good nofafional qualify groups) ANOVA wifh fime faken fo 
complefe fhe questions as fhe dependenf variable. We found a significanf 
main effecf for condifion, F(l, 36) = 4.71, p < .05, rf = .12. Regardless of 
nofafional qualify, parficipanfs took significanfly longer fo complefe fhe 
quesfions in fhe wrife condifion (good qualify: M = 8.6 min, SD = 3.6; 
poor qualify: M = 7.6 min, SD = 3.4) compared fo fhe no wrife condifion 
(good qualify: M = 6.7 min, SD = 2.5; poor qualify: M = 6.7 min, SD = 2.3). 
No ofher significanf effecfs were found. 

ART scores 

The mean score of parficipanfs on fhe ART was 11.3 (SD = 7.2) wifh a 
range of 2 fo 31. We computed correlation coefficienfs befween fhe ART 
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and quality of notations, number correct in the write condition, number 
correct in the no write condition, time to complete write condition and 
time to complete the no write condition. No significant correlations were 
found, suggesting that exposure to print was not related to the quality of 
notes produced by participants or their performance on the reasoning 
task. 

DISCUSSION 

In this study, we explored the effect of participants' production of 
different types of notations on their performance on analytical reasoning 
problems. As expected, the majority of participants made notes when 
they were given the opportunity. As Cary and Carlson (2001) have 
noted, creating notes is influenced by the costs and benefits of using 
them. The act of making notes takes time and effort as reflected by 
participants taking significantly longer to complete questions in the 
write condition than in the no write condition. It could be argued that 
participants in the present study judged the time and effort required in 
making notes to be outweighed by the benefits of using notes in solving 
the analytical reasoning problems. Alternatively, participants may have 
felt obligated to make notes in the write condition because they were 
provided with note-taking material in contrast to the no write condition 
where they were not allowed such materials. Nonetheless, previous 
research examining the more spontaneous production of notes also 
found that the majority of participants chose to make notes (Cox & Brna, 
1995; Eskritt et al., 2001). 

We predicted that making notes would aid in participants' reasoning 
skills in the write condition. The influence of notations, however, was 
not straightforward. Participants who made notes could be divided into 
those who made poor quality or good quality notes. The poor notational 
quality group was not as accurate in answering the analytical questions 
when given the chance to make notes compared to good notational 
quality group. However, this difference was not the result of good 
notational quality group doing better in the write condition compared to 
the no write condition. Instead, poor notational quality group tended 
not to perform as well in the write condition as compared to the no write 
condition, though this difference was not statistically significant. It is 
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important to note that no significant differences were found befween 
good and poor nofafional qualify groups in fhe no write condition 
implying that performance differences in fhe wrife condition were nof 
relafed fo general reasoning abilifies. Because fhere was also no 
significanf difference befween fhese fwo groups on fime faken fo 
complefe fhe quesfions, lengfh of fime carmof be fhe explanation. These 
findings suggesf fhaf fhe production of nofafions may nof always be 
beneficial. Parficipanfs' nofafional production could even be viewed as 
defrimenfal fo performance in fhe presenf sfudy because parficipanfs 
fook significanfly more fime fo complefe fhe problems in fhe wrife 
condifion fhan in fhe no wrife condition. 

Several possible factors explain why making notes did nof appear fo 
aid performance alfhough fheorisfs have argued fhaf exfemal 
represenfafions can influence reasoning (Cary & Carlson, 1999; Cox, 
1999; Zhang, 1997; Zhang & Norman, 1994). The facf fhaf even fhe good 
nofafional qualify group did nof benefif from producing notes was a 
particularly surprising finding. Alfhough fhe good nofafional qualify 
group had beffer qualify notes compared fo fhe poor nofafional qualify 
group, fheir nofes could have included more information. The mean 
score for fhe qualify of nofafions in fhe good nofafional qualify group 
was 2.2 (SD = .6) ouf of a maximum score of 4 and, as is apparenf in 
Figure 1, fhe mode was a score of 2. For many in fhe good nofafional 
qualify group, fheir nofes only re-represenfed fhe informafion confained 
in fhe question and did nof go beyond. Perhaps sfudenfs wifh nofes 
receiving a score of 3 or 4 may have found fheir nofes more beneficial in 
solving fhe problems; however, too few sfudenfs did so fo examine fhis 
possibilify. If has been nofed fhaf background knowledge abouf fhe use 
of diagrams is a predictor of success in reasoning wifh diagrams (Cox, 
1999). The adequate represenfafion of fhe informafion from fhe quesfion 
and consfrucfing an appropriate represenfafion for fhe specific fask are 
facfors fhaf impacf reasoning wifh nofes (Cox, 1999). Zhang and 
Norman (1994) also nofed fhaf differenf represenfafions of problems 
could have a large impacf on perceived problem difficulfy and 
participanf performance. The more fypes of represenfafions a parficipanf 
is aware of increases fhe number of possible represenfafions fhey can 
consfrucf fo solve a problem and increases fheir chance of picking one 
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that accurately represents the information necessary to solve the problem 
in the most efficient way possible (Cox, 1999). Therefore, parficipanfs 
may nof have had enough background knowledge abouf fhe use of 
diagrams and external representations to produce more effective 
notations. 

The ART was used as a measure of familiarify wifh differenf 
nofafional sysfems and fherefore if was predicfed fhaf fhe parficipanfs' 
level of prinf exposure would be posifively correlafed wifh qualify of 
nofes and performance on fhe fask. Unforfunafely, fhe ART was nof 
found fo correlafe wifh fhese measures. The ART is designed fo examine 
parficipanfs' exposure fo fexf because if measures how much reading fhe 
participanf engages in by fheir recognition of popular aufhors. Thus if 
may nof be an effecfive means of esfimafing parficipanfs' exposure fo 
ofher types of exfernal represenfafions such as matrices, which would be 
much more helpful for fhe reasoning problems presenfed in fhe presenf 
sfudy (Schwarfz, 1971; Schwarfz & Faffaleh, 1972). Therefore, fufure 
research needs fo use ofher measures fo examine familiarify of nofafional 
sysfems in general and its influence on fhe producfion of nofes in 
reasoning tasks. 

A number of factors besides background knowledge of exfernal 
represenfafions may also confribufe fo fhe difference in performance 
befween good and poor nofafional qualify groups in fhe write condifion. 
Because no significanf differences occurred befween groups in fhe no 
write condition, it is unlikely that participants varied on factors that can 
influence problem solving in general such as verbal abilify or working 
memory. However, variables such as learning sfrafegies and confrol 
processes have also been found fo influence nofafional producfion 
(Kiewra, 1988). Research is needed fo examine how fhese variables as 
well as background knowledge may inferacf fo influence parficipanfs' 
producfion and use of nofafions. 

The wifhin subjects comparison of performance in fhe wrife and no 
wrife condifions confrols for cognifive variables fhaf may explain 
individual differences in performance across fhe groups. The resulfs of 
fhe comparison across condifions indicafe fhaf nofafional production, 
regardless of qualify, did nof aid reasoning abilify. The difficulty of 
learning fo use symbol sysfems is offen underesfimafed in educafional 
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settings (Lehrer & Schauble, 2002). Students need to be exposed to the 
basic roles of external representations and they should have exposure to 
a wide range of exfemal represenfafions fhat are nof limifed fo a specific 
area of sfudy (i.e., physics, mafhematics). This exposure may lead fo an 
increase in sfudenfs' familiarify wifh represenfahons, allowing fhem fo 
apply nofations more effecfively. Differences in fhe performance of 
parficipanfs in fhe present study may also be due to differences in fheir 
educahonal experiences. Sfudenfs might differ in whaf fypes of 
represenfafions they know how to construct and apply. Furthermore, 
participants may differ in fheir previous experience using nofafions fo 
solve problems. Spending adequafe fime feaching sfudenfs to construct, 
interpret, and effectively utilize different forms of exfernal 
represenfafions is necessary. If should nof be assumed fhaf sfudenfs 
learn fhese skills wifhouf insfrucfion and pracfice (De Simone, Schmid, & 
McEwan, 2001). 
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APPENDIX - SAMPLE PROBLEM 

Below is a sample problem given to participants adapted from 
Brownstein et al. (1990). 

A professor offers a six-week course on film genres. Each time fhe 
following six genres are covered: advenfure films, defective films, 
fanfasy films, horror films, silenf films and wesferns. 

Wesfems are always covered before advenfure films. 

Defective films are always covered fhe week immediafely 
preceding, or fhe week immediafely following advenfure films. 
Horror films are covered eifher fhe firsf or fhe lasf week. 

Question 1: Which of fhe following is an accepfable schedule? 

a) fanfasy films, wesferns, advenfure films, silenf films, defective films, 
and horror films 

b) horror films, advenfure films, defective films, wesfems, fanfasy 
films, and silenf films 

c) horror films, wesferns, fanfasy films, advenfure films, silenf films, 
and defective films 

d) wesfems, fanfasy films, silenf films, advenfure films, defective films, 
and horror films 

e) advenfure films, defective films, silenf films, wesfems, fanfasy films, 
and horror films 

Question 2: If silenf films are shown in week 1 and wesferns are shown 
in week 2, whaf week(s) can advenfure films not be shown? 

a) 6 

b) 5,6 

c) 4,5,6 

d) 3 

e) 4,5 




